Stability fixes - #1574

Reading operational data can be slow, or time out entirely, while mDNS
neighbors are being discovered:

    statd[3658]: mdns: sr_apply_changes: Timeout expired
    statd[3658]: Error, getting operational data: User callback failed

statd is single threaded; the avahi handlers apply datastore changes
per resolver event, from the same event loop that serves all
operational get callbacks.  Every apply blocks the loop, and resolver
events arrive in bursts, e.g., after an mDNS daemon restart.

Batch all edits with a debounce timer and apply once the burst has
settled.  On datastore contention, back off and retry later instead of
blocking the loop.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>

A configuration change that is interrupted, e.g., by a frontend
timeout, can leave interfaces behind in the kernel: the datastore
transaction is rolled back but the dagger generation's side effects
are not.  The next generation then fails creating the interface:

    dagger[2599]: Aborting: /run/net/131/action/init/br0/50-init.ip failed with exitcode 1
    confd[2599]: Failed to apply interface configuration

with 'RTNETLINK answers: File exists' in the script log.  A failed
generation is abandoned, but its executed actions remain, so every
subsequent interface change fails, in either the create or the delete
direction, until reboot.

Delete any leftover before creating a netlink interface, and tolerate
an already-missing interface when deleting one.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>

Every configuration change that restarts the mDNS daemon logs a NOTICE
level reconnect message.  Routine noise, demote to INFO; the WARN/NOTE
pair for an unresponsive daemon is kept as is.

Also, minor code refactor included, dropping code useless block.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>

statd logs everything, including INFO level messages, to syslog by
default, unlike other services which default to NOTICE.

Port option handling from confd: -h, -V, and -v <level>, backed by
setlogmask().  Drop the stale flags from statd.conf, they were never
parsed and are rejected now that statd has strict option handling.

Unlike confd, -v info maps to LOG_INFO rather than LOG_NOTICE, and
-v debug also enables the DEBUG() macro without requiring the DEBUG
environment variable.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>

Configuring the system, or querying status, can be slow or time out
entirely while the periodic (5 min) journal snapshot is running.  On
slow systems with a big configuration a snapshot can take minutes.

The snapshot ran in a statd thread, reading ALL operational data in a
single sr_get_data("/*") call: every subsystem's callbacks are invoked
back-to-back while datastore locks are held for the entire read, with
statd's main loop busy serving them.  Other datastore users queue up
behind it.

Fork the snapshot instead, renamed statd-journal using prctl(), running
at nice 10 with its own sysrepo connection.  The dump is chunked, one
module per sr_get_data() call with a short breather in between, so
interactive users interleave with the snapshot instead of waiting for
all of it.  The snapshot timer is one-shot, re-armed only when the
previous snapshot has finished: snapshots can never overlap, and the
interval is the rest between them rather than a fixed cadence.

Each module read waits at most ten seconds: reading operational data
holds the module's read lock, also while waiting for providers, so on
a busy system the journal skips that module for the round instead of
holding up configuration changes:

    Failed sending edit-config RPC: Locking a rwlock failed (sr_shmmod_lock:
    Connection timed out), read lock held by running process 28334 (CID 85),
    running process 3627 (CID 3).  Retrying ...

Skipped modules are counted in the snapshot completion log message.

Also drops the last thread from statd, and the second sysrepo session,
which was only used by the journal.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>

A failing yanger invocation logged only:

    statd[3539]: Error, running yanger
    statd[3539]: Error adding interface yanger data

leaving no trace of which model or interface failed, or how.  Include
the model, the interface for per-interface queries, and yanger's exit
code.

Also plug a small memory leak: the extracted interface name was never
freed after the query.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>

Minimal images log the following on every boot and initctl reload,
e.g., when the regression test framework reconfigures the system:

    finit[1]: Skipping /etc/finit.d/enabled/netbrowse.conf, dangling symlink: No such file or directory
    finit[1]: service_register():/etc/finit.d/enabled/ttyd.conf: skipping ttyd: No such file or directory
    finit[1]: Skipping /etc/finit.d/enabled/webui.conf, dangling symlink: No such file or directory

The web services are enabled in the default configuration, so confd's
finit_enable() creates enabled/ symlinks also on images where the
service was never installed.  svc_enable() already guards its nginx
symlinks with the corresponding check.  The ttyd case differs: its
conf ships unconditionally in the common rootfs skeleton while the
daemon itself is an optional package.

Skip enable, with a log message at INFO, when the service conf is not
available in the image.  At build time, drop the ttyd confs when ttyd
is not selected, and prune any dangling enabled/*.conf symlinks.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>

-v info mapped to LOG_NOTICE, making actual INFO level messages
unreachable from the command line.  -v debug opened the log mask but
not the DEBUG() macro, which is gated on the debug variable, so debug
messages still required the DEBUG environment variable to be set.

Same behavior as statd.

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>